Uncertainty and Error Handling in Pervasive Computing: A User’s Perspective

From this chapter’s perspective, pervasive computing is a new class of multimodal systems, which employs passive types of interaction modalities, based on perception, context, environment and ambience (Abowd & Mynatt, 2000; Feki, 2004; Oikonomopoulos et al., 2006). By contrast, early multimodal systems were mostly based on the recognition of active modes of interaction, for example speech, handwriting and direct manipulation. The emergence of novel pervasive computing applications, which combine active interaction modes with passive modality channels raises new challenges for the handling of uncertainty and errors. For example, context-aware pervasive systems can sense and incorporate data about lighting, noise level, location, time, people other than the user, as well as many other pieces of information to adjust their model of the user’s environment. In affective computing, sensors that can capture data about the user’s physical state or behaviour, are used to gather cues which can help the system perceive the user’s emotions (Kapoor & Picard, 2005; Pantic, 2005). In the absence of recognition or perception error, more robust interaction is then obtained by fusing explicit user inputs (the active modes) and implicit contextual information (the passive modes). However, in the presence of errors, the invisibility of the devices that make up the pervasive environment and the general lack of user’s awareness of the devices and collected data properties render error handling very difficult, if not impossible. Despite recent advances in computer vision techniques and multi-sensor systems, designing and implementing successful multimodal and ubiquitous computing applications remain difficult. This is mainly because our lack of understanding of how these technologies can be best used and combined in the user interface often leads to interface designs with poor usability and low robustness. Moreover, even in more traditional multimodal interfaces (such as speech and pen interfaces) technical issues remain. Speech recognition systems, for example, are still error-prone. Their accuracy and robustness depends on the size of the application’s vocabulary, the quality of the audio signal and the variability of the voice parameters. Signal and noise separation also remains a major challenge in speech recognition technology. Recognition-based multimodal interaction is thus still error prone, but in pervasive computing applications, where the capture and the analysis of passive modes are key, the possibilities of errors and misinterpretations are even greater. Furthermore, in pervasive